Evaluation of Terminology Extractors: Principles and Experiments

We claim that the evaluation of terminology extraction systems cannot rely entirely on the comparison of their results with a reference list, as the very existence of such a list made up independently of any application is highly questionable and as these systems do not yield only actual terms but potential (or candidate) terms. We propose instead to align their results on various corpora, so as to isolate the divergences and the convergences and to understand their rationale. This paper compares two noun phrase (NP) extraction softwares. It proved necessary to translate the NP parses produced by these softwares into a pivot grammar. We developed a Frontier to Root Transducer based on syntax-directed translation schemas used for standardization and optimization in the compiling process. These tried and tested compiling techniques provide an efficient and nevertheless declarative way of obtaining mappings between very different parses ("skeleton parses" / detailled constituent structures) and to evaluate the underlying